Image rendering in computer graphics has been a continually evolving field, with more and more techniques being developed as time goes on and computer technology progresses. Prior rendering approaches often involved constructing or recovering a complete geometric and photometric model of a 3D scene. However, these approaches are typically complex and computationally intensive. One image-based rendering approach that foregoes a need for a geometric/photometric model of a scene uses a plenoptic function to describe the scene as previously described by Adelson and Bergen in “The plenoptic function and the elements of early vision”, Computational Models of Visual Processing, pp. 3-20. MIT Press, Cambridge, Mass., 1991.
The original Adelson and Bergen work defined a 7D plenoptic function as the intensity of light rays passing through the camera center at every location (Vx, Vy, Vz) at every possible angle (θ, φ), for every wavelength λ, at every time t, i.e.,P7=P(Vx,Vy,Vz,θ,φ,λ,t)  (1)
In recent years a number of image-based rendering techniques have been proposed to model and then render real or synthetic scenes and objects based on attempts to simplify the plenoptic function. For example, McMillan and Bishop have proposed constructing a complete 5D plenoptic function in “Plenoptic modeling: An image-based rendering system,” Computer Graphics (SIGGRAPH'95), pp. 39-46, August 1995:P5=P(Vx,Vy,Vz,θ,φ)  (2)
In this prior work, two of the variables in the original equation (equation (1)) are dropped, namely time t and light wavelength λ. This approach assumes a static environment having fixed light conditions.
A 4D parameterization of the plenoptic function has also been proposed by M. Levoy and P. Hanrahan in “Light field rendering,” Computer Graphics Proceedings, Annual Conference Series, pp. 31-42, Proc. SIGGRAPH'96 (New Orleans), August 1996, and S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen in “The Lumigraph,” Computer Graphics Proceedings, Annual Conference Series, pp. 43-54, Proc. SIGGRAPH'96 (New Orleans), August 1996. With both of these systems, by staying outside a convex hull or bounding box of an object, the 5D complete plenoptic function can be simplified to a 4D light field plenoptic function, i.e.,P4=P(u, v, s, t)  (3)where (u, v) and (s, t) parameterize two bounding planes of the convex hull.
There have even been 2D simplifications proposed, such as cylindrical panoramas by S. Chen in “QuickTime VR,” Computer Graphics Proceedings, Annual Conference Series, pp. 29-38, Proc. SIGGRAPH'95, August 1995, and spherical panoramas by R. Szeliski and H. Shum in “Creating full view panoramic image mosaics and texture-mapped models,” Computer Graphics Proceedings, Annual Conference Series, pp. 251-258, Proc. SIGGRAPH'97, August 1997, where the viewpoint in the scene is fixed and only viewing directions can be changed, i.e.,P2=P(θ,φ)  (4)
The 2D embodiment of the plenoptic function is the easiest to construct. However, the 2D parameterization of the plenoptic function does not allow novel views from different viewpoints within the scene to be rendered. Although it would be possible to render novel views using the 5D or 4D embodiments of the plenoptic function, it is very time and storage consuming to construct a 5D complete plenoptic function. In addition, the prior 4D embodiments are limited to looking at a small object from the surrounding environment (i.e., in an “outside-looking-in” situation), rather than looking around the outside environment (i.e., in an “inside-looking-out” situation).
More recently, a 3D plenoptic function has been proposed by Heung-Yeung Shum and Li-Wei He in “Rendering with concentric mosaics,” Computer Graphics Proceedings, Annual Conference Series, pp. 299-306, Proc. SIGGRAPH'99, August 1999. Their method allows an observer to move freely within a 2D circular region and observe lateral parallax and lighting changes without geometric or photometric scene model recovery. However, situations can arise where the method has difficulty correctly reproducing rays off the capture plane, resulting in vertical distortions.
The scene capturing and view rendering based on a longitudinally aligned camera array described below addresses these and other disadvantages.